Region detection and obscuring logic for generating privacy protected images

ABSTRACT

Systems and methods are disclosed and an example system includes a digital image receiver for receiving a digital image, and an automatic obscuration processor coupled to the image receiver and configured to determine whether the digital image includes a region that classifies as an image of a category of object and, upon a positive determination, to obscure the region and output a corresponding obscured-region digital image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit under 35 U.S.C. § 120 to U.S. patentapplication Ser. No. 17/213,851 filed Mar. 26, 2021, which claims thebenefit of priority under 35 U.S.C. § 119(e) to U.S. Provisional PatentApplication No. 63/005,861, filed Apr. 6, 2020, entitled “Privacy-AwareCapture and Device,” the disclosures of both of which are herebyincorporated by reference in their entirety.

STATEMENT OF GOVERNMENT INTEREST

The present invention was made by employees of the United StatesDepartment of Homeland Security in the performance of their officialduties.

FIELD

The present disclosure relates generally to privacy aspects of imagecapture.

BACKGROUND

Image capture devices such as video cameras can promote public safetyand security. However, some applications may present arguable concernsas to privacy. Some concerns may arise from risk, or perceived risk, ofunauthorized access to or distribution of feeds from image capturedevices. Such concerns or perceptions can be elevated for image capturedevices that due to desired of meeting the devices' purposes of publicsafety and security, capture personal identifiable information (PII).Examples of PII that can be captured can include, but are not limitedto, the geometry and other features of persons' faces, automobilelicense plate numbers, and personal name tags.

SUMMARY

Systems are disclosed and one example can include a digital imagereceiver configured to receive a digital image, and an automaticobscuring processor, coupled to the image receiver configured todetermine whether the digital image includes a region that classifies asan image of a category of object and, upon a positive determination, toobscure the region and output a corresponding obscured-region digitalimage.

Methods are disclosed and one example can include receiving a digitalimage, determining whether the digital image includes a region thatclassifies as an image of a category of object and, upon a positivedetermination obscuring the region and outputting a correspondingobscured-region digital image.

Other features and aspects of various embodiments will become apparentto those of ordinary skill in the art from the following detaileddescription which discloses, in conjunction with the accompanyingdrawings, examples that explain features in accordance with embodiments.This summary is not intended to identify key or essential features, noris it intended to limit the scope of the invention, which is definedsolely by the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawing figures illustrate one or more implementations in accordingwith the teachings of this disclosure, by way of example, not by way oflimitation. In the figures, like reference numbers refer to the same orsimilar elements. It will be understood that the drawings are notnecessarily to scale.

FIG. 1 shows a simplified functional block schematic of an examplesystem for privacy-aware image capture in accordance with the presentdisclosure.

FIG. 2 shows a logic flow chart for operations in a process in one ormore methods for privacy-aware image capture in accordance with thepresent disclosure.

FIG. 3 shows a logic flow chart for various operations in anotherprocess in one or more methods for privacy-aware image capture inaccordance with the present disclosure, which includes an implementationof a multiple category obscuring feature.

FIG. 4 shows a simplified functional block schematic of another systemfor privacy-aware image capture in accordance with the presentdisclosure, employing aspects of the FIG. 1 block schematic, and showingone configuration for the identity-correlated object obscuring logic.

FIG. 5 shows a logic flow chart for various operations in anotherprocess in privacy-aware image capture in accordance with the presentdisclosure, including example operations in an implementation of theFIG. 4 identity-correlated object obscuring logic.

FIG. 6 shows a functional block schematic of another system forprivacy-aware image capture in accordance with the present disclosure,providing an integrated image capture/identity-correlated object detectand obscuration.

FIG. 7 shows a simplified functional block diagram of another system forprivacy-aware image capture in accordance with the present disclosure,which includes a key-access reversible obscuring of variousidentity-correlated objects, with key management.

FIG. 8A shows example aspects of a random subject, including an examplehead and a face, positioned above an abstracted torso, the face being anexample identity-correlated object for subsequent processing.

FIG. 8B shows an example object obscuring in accordance with the presentdisclosure, corresponding to the FIG. 8A face.

FIG. 8C shows an example replacement privacy preserving image, with anobscuring of the face, resulting from modifications of the FIG. 8Aimage, based at least in part on the FIG. 8B obscuring.

FIG. 9A shows aspects of a random subject, including a face above anabstracted torso, and a nametag on the subject's shirt.

FIG. 9B shows example obscuring regions, corresponding to the FIG. 9Aface and nametag.

FIG. 9C shows a replacement privacy preserving image, resulting frommodifications of the FIG. 9A image, based at least in part on the FIG.9B example obscuring regions corresponding to the FIG. 9A face andnametag.

FIG. 10A illustrates a capturing of an image of an automobile, withexample identity-correlated objects, on a privacy aware image capturesystem in accordance with the present disclosure.

FIG. 10B illustrates a privacy preservation replacement of the FIG. 10Aimage, providing category-specific obscuring of identity-correlatedobjects, via methods in accordance with the present disclosure.

FIG. 11 shows a simplified functional block schematic of a computersystem on which aspects of systems and methods in accordance with thepresent disclosure can be practiced.

DETAILED DESCRIPTION

Aspects and features of disclosed systems and methods for privacyprotection image capture include, but are not limited to, automaticdetection of instances, in a digital image captured by a digital camera,of identity-correlated objects and, in response, outputting aprivacy-protected image that obscures regions detected as having anidentity-correlated objects. Embodiments can include, among otherprivacy protective features, no output of the originally captured image,and not storing or maintaining the originally captured image nor anycopy of that image is stored or maintained.

For simplicity, “region(s) of an image detected as likely including anidentity-correlated object” and variations thereof may be referred to as“identity-correlated regions.”

Embodiments can feature irreversible obscuring of identity-correlatedregions. Embodiments can feature reversible obscuring, and combinationsof reversible and irreversible obscuring. Embodiments can be configuredto detect what may be a library of identity-correlated objects.Embodiments can feature generic, single-category detection, andobscuring. Various embodiments can include classification ofidentity-correlated regions into one among a plurality of categories.Example categories of identity-correlated objects can include, withoutlimitation, persons' faces, tattoos, name tags, and license platenumbers. One or more embodiments can feature category-specificobscuring, e.g., using respectively different obscuration types ondifferent categories of identity-correlated regions.

Embodiments can feature detection of a particular pattern or arrangementof objects. The objects forming the particular pattern, standing alone,may be identity-correlated objects. One more of the objects forming theparticular pattern may be a non-identity correlated category of object.In embodiment features can include at least partially obscuring theparticular pattern.

One more embodiments can provide, for captured moving picture images,digital processing resource conservation features, for example, withoutlimitation, applying identity-correlated object detection to 1 out of Rof the sequence of digital image frames (hereinafter “frames”) of an Mframe-per-second image, with R and M being integers. Implementations caninclude detecting of moving objects, estimating of location withinsubsequent frames of detected moving objects and corresponding real-timeobscuring utilizing estimated locations.

Embodiments can feature a plurality of different key-based reversibleobscuring processes, which can be selected from and applied based on thedetected category of identity-correlated regions. Embodiments canfurther a key management feature, in which certain entities can beprovided decryption keys to certain among the different key-basedreversible obscuring processes.

Embodiments can include a security housing that encloses a photodetectorpixel array coupled to a particularly configured processing resource,the configuration providing within the security housing privacy-awareobscuring of identity-correlated regions, and corresponding transmissionout from the housing of a replacement, privacy preserving image.

FIG. 1 shows a simplified functional block schematic of a system 100that can provide, among other features, privacy-aware image capture andvarious aspects thereof in accordance with the present disclosure.Implementations of the system 100 can include an identity-correlatedregion obscuring logic 102, which can be configured to receive, e.g.,from an image capture device 104, an image (labelled “IMG” in thefigure) of a subject, such as the example person labeled in the figureas “SB”). It will be understood that “IMG” and “SB” are arbitrary labelshaving no intrinsic meaning.

Embodiments can feature identity-correlated region obscuring logic 102configured to detect identity-correlated regions using one or morelikelihood threshold-based classification processes. The numerical valueof “likely” can be application-specific. Factors can includeapplication-specific ranges of acceptability of false positive and offalse negative.

Regarding object detection configuration of the identity-correlatedregion obscuring logic 102, in an embodiment logic 102 can utilize, asone example, the Viola Jones algorithm. Published descriptions of theViola Jones algorithm are readily available, and persons of ordinaryskill in the relevant arts, upon reading the present disclosure in itsentirety, can adapt the published algorithm to practices in accordancewith the present teachings. Detailed description of the Viola Jonesalgorithm is therefore omitted. Of academic interest, an exampledescription can be found in H. Viola and M. Jones, Robust Real-TimeObject Detection, Second International Workshop on Statistical andComputational Theories of Vision—Modeling, Learning, Computing, andSampling, July 2001.

Another example implementation of the object detection configuration ofthe identity-correlated region obscuring logic 102 theKanade-Lucas-Tomasi (KLT) object tracking algorithm to detect, classify,and determine location of privacy-correlated regions. Publisheddescriptions of the KLT algorithm are readily available, and persons ofordinary skill in the relevant arts, upon reading the present disclosurein its entirety, can adapt the KLT algorithm to practices in accordancewith the present teachings. Detailed description of the KLT algorithm istherefore omitted. Of academic interest, an example description is givenin R. Shi and C, Tomasi, Good Features to Track, IEEE Conference onComputer Vision and Pattern Recognition, 1994.

In an embodiment, the identity-correlated region obscuring logic 102 canbe configured to use one or more neural network object detection andclassification processes. Examples include, but are not limited to,region-based Convolutional Neural Networks (R-CNN), Fast R-CNN, andFaster R-CNN. Published descriptions of the R-CNN, Fast R-CNN, andFaster R-CNN are readily available and persons of ordinary skill in therelevant arts, upon reading the present disclosure in its entirety, canadapt any one or more of the above-identified published CNN algorithmsto practices in accordance with the present teachings. Detaileddescription of the example CNN algorithms, R-CNN, Fast R-CNN, and FasterR-CNN is therefore omitted. Of academic interest, an example descriptionof R-CNN can be found in Uijlings, J. R., et al., Selective Search forObject Recognition, International Journal of Computer Vision 104.2(2013); an example description of Fast R-CNN can be found in R.Girshick, Fast R-CNN, Microsoft Research, 2015; and an exampledescription of Faster R-CNN can be found in S. Ren, et al., FasterR-CNN: Towards 14 Real-time Object Detection with Region ProposalNetworks, Neural Information Processing Systems (NIPS), 2015.

Regarding obscuring process configurations of the identity-correlatedregion obscuring logic 102, the logic can be configured to obscuredetected identity-correlated regions via one or more types of reversibleobscuring, or via one or more types of irreversible obscuring. Theidentity-correlated region obscuring logic 102 can be configured toapply category-specific selected obscuring, for example, among differenttypes of reversible obscuring, different type of irreversible obscuring,or both.

Features of the identity-correlated region obscuring logic 102 caninclude non-reversible obscuring and can include non-reversible orsubstantially non-reversible blurring, for example, digital blurring, orpixilation. The obscuring can render the obtaining of useful privacyrelated data to be computationally impractical. Obscuring processesprovided by the logic 102 can be configured to irreversibly remove ordestroy the identity-related information content from all of thedetected identity-correlated region, or from certain categories of suchregions.

The identity-correlated region obscuring logic 102 can be configured toobscure via distortion. The logic 102 can be configured to applyreversible distortion, or irreversible distortion, or to select betweensuch types based on category. Processes of reversible distortion andirreversible distortion applied by the identity-correlated regionobscuring logic 102 can include permanent replacement of the originaldata, e.g., the privacy related data, with new data. Theidentity-correlated region obscuring logic 102 can be configured togenerate the new data as a reversible algorithmic distortion, e.g.,key-based encryption, of the original data. The logic 102 can beconfigured to generate the new data by an irreversible distortion, e.g.,pixilation or Gaussian blurring of the original data. In an embodiment,the identity-correlated region obscuring logic 102 can be configuredgenerate, in a process of irreversible obscuring, the new data as asynthesized object. The generation can use, for example, a generativeadversarial network (GAN) technique. One example can be, but is notlimited to, the StyleGAN system available from Nvidia, Inc., 2788 SanTomas Expy, Santa Clara, Calif. 95051.

The identity-correlated region obscuring logic 102 can be configured toobscure via generating new data via a process independent of the data inthe identity-correlated region. On example can be making data, e.g.,replacing all pixels in the region with a pre-defined mask.

In accordance with one or more embodiments, both for non-reversibleobscuring and reversible obscuring, the original data can be discarded,e.g., all instances erased from processor memory.

In one or more embodiments, configuring the identity-correlated regionobscuring logic 102 can include determining an acceptable spatialresolution, e.g., details of boundary contour, of identity-correlatedregions as detected by the logic 102. Related to this, a configuring ofthe identity-correlated region obscuring logic 102 can includedetermining an acceptable accuracy in spatial registration, i.e.,location and orientation, between identity-correlated regions asdetected by the logic 102, and the actual location and orientation ofthe region in the image. One reason is that logic 102 obscuringprocesses, in an aspect, obscure identity-correlated regions as detectedby the logic 102. Mis-registration and other spatial differences ofbetween such regions and the identity-correlated regions as they appearin the original image can result in obscuring more or obscuring lessthan the original identity-correlated region. This can be a factor inselecting and configuring the object detection and localizationalgorithm(s) to include in the identity-correlated region obscuringlogic 102, because different object detection and localizationalgorithm(s), including different ones among the example algorithmsidentified above, can exhibit different performance with respect toobject detection boundary and localization.

TABLE 1 Alteration Remediation Strategies Strategies Capture-BasedRemediation Algorithm-Based Remediation Reversible Privacy related dataPost-processing of media distortion protected through distortiondistorts privacy data, but by edge processing; with authorized key usecan authorized use key can be restore subsequently be restoreIrreversible Permanent distortion Post-processing of media distortionintroduced by edge permanently distorts PII processing

The image IMG received from image capture device 104 can be a stillimage, or can include a sequence of images, i.e., a moving pictureimage. A moving picture IMG can be an event triggered, e.g.,motion-triggered image capture. In one more embodiments, theidentity-correlated region obscuring logic 102 can include technicalfeatures that provide, among other benefits, reduced computationprocessing of moving picture IMGs. Technical features can enable realtime privacy-protective various frame rates of moving picture IMGs, andexamples can include or encompass a range extending from approximately20 frames per second (fps), or perhaps less than 20 fps, up to, forexample, 100 fps or higher. These are only examples, but in variousapplications applying the same identity-correlated region detection andclassification process, and same obscuring process to each frame maycarry an undesirable computation burden. In an embodiment theidentity-correlated region obscuring logic 102 can be configured toapply a faster, less accurate object detection, localization, andclassification processes to some frames, and apply less often a moreaccurate object detection, localization, and classification processes(e.g., every 10th frame). In an embodiment, if an identity-correlatedregion is detected in the frame, an obscuration process can obscure acorresponding region in number of frames succeeding and a number offrames preceding the frame.

In an embodiment, the identity-correlated region obscuring logic 102 canbe configured to receive compressed moving picture IMGs, e.g., H.264(also referred to as MPEG-4), H.265 format, from the image capturedevice 104. The logic 102 can be further configured to detectidentity-correlated regions within the compressed moving picture IMG,and to output a compressed moving picture privacy-protected image.

As understood by persons of ordinary skill in the pertinent arts,compression algorithms such as H.264, H.265, and others compress byremoving certain frame to frame image redundancy. Operations includereplacing sequences of full frames with what may be termed a spacedsequence, as it includes full frames spaced apart by image-changeframes. Published description of the H.264 and H.265 formats are readilyavailable and therefore detailed description is omitted from thisdisclosure. Of academic interest, an example description of H.264 can befound in D. Marpe, et al., The H.264/MPEG4 Advanced Video CodingStandard and Its Applications, IEEE Communications Magazine, September2006.

Operations in various implementations of protection processes providedby one or more embodiments can include certain interface with, forexample, the H.264 and H.265 format. In these formats, and others, fullframes are referred to as “I” frames and the image-change frames arereferred to as “P” frames and “B” frames. In one embodiment, features ofsystem 100 can include a compression coder-decoder (CODEC) that canconvert the compressed moving picture IMG to an uncompressed format, andthen via the identity-correlated region obscuring logic 102, perform theabove-described detection and obscuration of identity-correlatedregions, followed compressing the output privacy-protected moving IMGback to the compressed format, e.g., H.264 or H.265. In anotherembodiment, the identity-correlated region obscuring logic 102 can beconfigured to apply the above-described detecting to I frames, generatenew data for the I frames' detected identity-correlated regions,generate P frame versions, or P frame and B frame versions of the newdata, and then output a privacy protected H.264 or H.265 moving pictureIMG. One or more embodiments can configure the identity-correlatedregion obscuring logic 102 to switch between a CODEC mode, i.e., decode,then apply detection and obscuring, then encode, and a compressed formatmode, i.e., operate directly on the compressed moving IMG.

It will be understood that the graphic blocks in FIG. 1 represent logicfunctions and do not necessarily define or limit hardware architecture.For example, the identity-correlated region obscuring logic 102 can beintegrated within the image capture device 104. In anotherimplementation, the identity-correlated region obscuring logic 102 andimage capture device 104 can within a security housing, to resist directaccess to the output of the image capture device 104. In anotherimplementation, a plurality of image capture devices 104 can beprovided, each configured to feed encrypted image data to a sharedresource implementing the identity-correlated region obscuring logic102. The shared resource implementing the identity-correlated regionobscuring logic 102 can be configured to receive and decrypt the outputsof all the image capture devices 104, apply the above-describedidentity-correlated region obscuring operations to each, and output acorresponding plurality of privacy-protected images. The shared resourceimplementation of the identity-correlated region obscuring logic 102 canfeature, as described above, no output and no storage ofidentity-correlated region data received from any of the plurality ofimage capture devices.

Implementations of the identity-correlated region obscuring logic 102can include, for example, a programmable processor that can be coupledto or can otherwise have access to an instruction memory. Theinstruction memory can include a tangible medium that can storeprocessor-executable instructions that can cause the processor toperform various processes and operations thereof, in methods describedin greater detail in later sections.

Referring to the FIG. 1 image capture device 104, it will be understoodthat the device 103 can be configured for image capture in variousspectra. Example spectra can include, without limitation, visible lightimaging, non-visible light imaging, e.g., one or more of the infra-redbands, and millimeter wave imaging. It will be understood that“image(s),” as used herein, can include visible light representations ortranslations of images originally captured in other spectra. The imagecapture device 104 can be configured to include light amplification(e.g., “night-vision”) imaging.

FIG. 2 shows a logic flow chart representing a flow 200 in operations inone or more methods for privacy-aware image capture in accordance withthe present disclosure. Description of operations in the flow 200includes references to FIG. 1 . The references are for purposes ofconvenience and are not intended to limit practices of the flow 200 tothe FIG. 1 system.

Upon receiving 202 a digital image, e.g., IMG from the FIG. 1 imagecapture device 104, the flow 200 can proceed to detecting and obscuring204. In an embodiment, operation in the detecting and obscuring 204 caninclude the identity-correlated region obscuring logic 102 applyingthreshold-based object detection of identity-correlated regions,generation of a new or replacement data for each of the regions andoutputting a privacy-protected image in which all detectedidentity-correlated regions are obscured. As described above, theidentity-correlated region obscuring logic 102 can be configured to useor apply, in its detecting of identity-correlated regions, operationsaccording to the Viola Jones algorithm, or operation according to a CNNalgorithm, such as R-CNN, Fast R-CNN, or Faster R-CNN, or variouscombinations thereof. As also described above, region obscuringoperations operation in the detecting and obscuring 204, e.g., by theidentity-correlated region obscuring logic 102, can include reversibleobscuring, e.g., public key encryption, or irreversible obscuring, e.g.,digital blurring, or pixilation. The obscuring can to an extent that,except for regions reversibly obscured by a process for which areceiving entity has authority to access, obtaining useful privacyrelated data can be computationally impractical.

As also described above, if the image received at 202 is a moving image,operations in the detecting and obscuring 204 may configured tointerface a compression process, e.g., H.264 or H.265, or others. In anembodiment, operations in the detecting and obscuring 204 may beconfigured to provide a compression process, such as but not limited to,H.264 or H.265, or others. For example, in an embodiment, detecting andobscuring 204 operations can be applied to every r^(th) frame. In ther^(th) frames, example operations can include detecting everyidentity-correlated region, and applying to such regions obscuringoperations, as described above. Such operations can include generating anew data for the region's location in the r^(th) frame. In anembodiment, for obscuring using, e.g., reversible encryption, detectingand obscuring 204 operations can generate a per-frame movement data, fora motion-compensated, shifted location version of the new data forobscuring the region in the intervening frames. Also, if the imagereceived at 202 is a compressed moving image, e.g., an H.264 or H.265format image, one or more embodiments can operate on the compressedimage, e.g., applying in the detecting and obscuring 204 operations ofdetecting and generating new data for identity-correlated regions in thefull or I frames, generating P frame versions of the new data, andoutputting a privacy protected H.264 or H.265 image, using the fullframe new data and the P frame versions of the new data. In an aspect,operations in the detecting and obscuring 204 can include generating Bframe versions of the new data.

The type of obscuring used in the detecting and obscuring 204 can beautomatically selected in accordance with the detected category. Forexample, the detecting and obscuring 204 can include automatic selectingamong a plurality of different, category-specific types of reversibleobscuring. In another example, the detecting and obscuring 204 can beconfigured to include an automatic category-specific selecting betweenirreversible obscuring and reversible obscuring. In an embodiment theobscuring processes in the detecting and obscuring 204 can becategory-specific selection between reversible and irreversible, as wellas selection among different types of reversible obscuring, or differenttypes of irreversible obscuring, or both. One example can includeselecting irreversible obscuring for regions determined likely to haveobjects of any category within one group, and reversible obscuring forregions determined likely to have objects of any category within anothergroup.

FIG. 3 shows a logic flow chart for a flow 300 of operations in anotherprocess in a privacy-preserving image capture in accordance with thepresent disclosure. The flow 300 assumes the above-identifiedconfiguration of the identity-correlated region obscuring logic 102 thatincludes categorizing of the identity-correlated regions andcategory-specific obscuring. The flow 300 also assumes theidentity-correlated region obscuring logic 102 is configured to detectinstances of identity-correlated information within any of N categories,N being a positive integer.

In an instance of the flow 300, operation can proceed from receiving 302an image file, e.g., an IMG from the FIG. 1 image capture device 104, todetecting 304 whether any identity-correlated regions are present in theimage. If the detecting 304 result is “No,” the flow 300 can return to302. In an aspect, if the detecting 304 result is “Yes” the flow 300 canproceed to categorizing 306, which can include determining which of theN categories into which the detected identity-correlated region fits. Itwill be understood that the FIG. 3 depiction of the detecting 304 andcategorizing 306 as separate blocks, and the sequential descriptionabove of the detecting 304 and categorizing 306 is not intended to limitthe detecting 304 and categorizing 306 to any grouping or sequencing.For example, as described above, in one or more implementations thedetermining of the category of an identity-correlated region categorywithin an image can be integral with detecting the region's presence.

The detecting 304 and categorizing 306 are not limited to a singleidentity-correlated region, and instead can include detecting aplurality of separate identity-correlated regions in the image. Also, inan embodiment, the identity-correlated region obscuring logic 102 can beconfigured to detect, and to apply commensurate obscuring to particularcombinations of identity-correlated regions. For example, a visiblefirst name brand logo or insignia on a first type of apparel, e.g., ashirt, in combination with a visible second name brand logo or insigniaon a second type of apparel, e.g., a headband, may be well enoughassociated with a particular person to be personal identity information.

Upon the categorizing 306 of detected identity-correlated regions in theimage, the flow 300 can proceed, for each detected identity-correlatedregions, to an n^(th) type obscuring 308-n of the region. The “n” can beaccording to the region's category. For example, upon detecting 304 andcategorizing 306 an instance of a first category identity-correlatedregion, the flow 300 can proceed to a first type obscuring 308-1 for theregion. Upon detecting 304 and categorizing 306 an instance of a secondcategory identity-correlated region, the flow 300 can proceed to asecond type obscuring 308-2 for the second category region. For purposesof description, the first type obscuring 308-1, second type obscuring308-2, . . . , n^(th) type obscuring 308-n can be collectivelyreferenced as “obscuring types 308.”

An example will assume a received image that includes two more instancesof first category identity correlated regions and two or more instancesof second category identity-correlated regions. The flow 300 can beconfigured to apply a first type obscuring 308-1 to each of the two ormore first category identity-correlated regions and to apply a secondtype obscuring 308-2 to each of the two or more second categoryidentity-correlated regions. The two instances of the first typeobscuring 308-1 are not necessarily mutually identical, and the twoinstances of the second type obscuring 308-2 are not necessarilymutually identical. For example, assuming the first type obscuring 308-1is a key-based encryption reversible obscuring, one instance of thefirst type obscuring 308-1 applied to one of the two or more firstcategory identity-correlated regions may use a different encryption keythan an instance of the first type obscuring 308-1 applied to another ofthe two or more first category identity-correlated regions.

As described above, in an embodiment the detecting 304 or thecategorizing 306, or both, can be configured to determine whether thedigital image includes a particular combination of regions thatindividually classify as an image of a category of object. An embodimentcan include, together with determining whether the digital imageincludes the particular combination of regions, a correspondingconfiguring of an obscuring or a combination obscuring. For example, thefirst type obscuring 308-1 or the second type obscuring 308-2 can beconfigured such that, in response to a positive determination by thedetecting 304 or the categorizing 306, or both, that the image includesa combination of regions that, together, constitute personalidentification information, one or more of the constituent regions canbe obscured. In one or more embodiments, detection of combinations ofregions constituting personal identification information can includedetecting particular positionings, e.g., patterns of the constituentregions. In such embodiments, the obscuring of one or more of theconstituent regions can include at least partial obscuring of theparticular positioning.

FIG. 4 shows a simplified functional block schematic of animplementation of another system 400 for privacy-aware image capture inaccordance with the present disclosure. The system 400 can receive animage data from an imaging device 402, such as but not limited to theFIG. 1 image capture device 104. In an embodiment, the system 400 caninclude an image buffer 404, an identity-correlated region detectionlogic 406, an obscuring logic 408, and a privacy protected image outputlogic 410. The image buffer 404 can be separate from or can beintegrated with the identity-correlated region detection logic 406, orobscuring logic 408. The identity-correlated region detection logic 406can be configured to detect one or more identity-correlated objectcategories. Example categories can include, but are not limited to,human faces, nametags, tattoos, and automobile license plate numbers.

In an embodiment, the identity-correlated region detection logic 406 canbe configured to determine, for each detected instance of an objectcategory, certain information about the instance. For purposes ofdescription, such information will be alternatively referred to as“category instance information.” Examples of category instanceinformation can include, without limitation, the instance's location,shape, geometry, dimension, or any combination or sub-combinationthereof. The identity-correlated region detection logic 406 can beconfigured to provide the category instance information to the obscuringlogic 408 or to the privacy protected image output logic 410, or both.

In an embodiment, obscuring logic 408 can be configured to generateobscuring data based at least in part on the above-described categoryinstance information from the identity-correlated region detection logic406.

In an embodiment, the obscuring logic 408 can be configured to generateobscuring data in a format and with an informational content that can bebased a plurality of factors. One example factor can be the specificdetected identity-correlated object category. Another factor can bewhether the obscuring is reversible or irreversible. For purpose ofdescription the factor of the specific detected identity-correlatedobject category can be referred to as a “first” factor. The first factormay be independent. The factor of whether the obscuring is reversible orirreversible can be referred to as a “second” factor. In an embodiment,the second factor can be dependent at least on part on the first factor.

In an embodiment, the obscuring logic 408 can be configured to generateobscuration data, for one or more categories, to function as selectorsor instructions for use by the privacy protected image output logic 410.For example, the obscuring logic 408 can be configured to includeencryption-based reversible obscuring of one or more categories ofidentity-correlated regions. In such implementation, the obscuring logic408 can be configured to generate, for the one or more categories ofidentity-correlated objects, category-appropriate encryption parametersand to provide the parameters to the privacy protected image outputlogic 410.

It will be understood that the respective graphic blocks representingthe image buffer 404, the identity-correlated region detection logic406, the obscuring logic 408, and the privacy protected image outputlogic 410 represent logic functions. Neither the per-block partitioningof functions and features, nor the arrangement of the graphic blocks isintended as any limitation of implementations to any specificarchitecture, technology, arrangement or geographic distribution ofhardware. For example, various combinations of, or all of the imagebuffer 404, identity-correlated region detection logic 406, obscuringlogic 408, and privacy protected image output logic 410 may beimplemented using a processor having access to an instruction memorythat can store, in a tangible medium, processor-executable instructionsthat can cause the processor to perform the logic blocks' describedfunctions.

For example, a contemplated implementation can include merging theidentity-correlated region detection logic 406 and the obscuring logic408, into a merged logic. An implementation of the merged logic may alsofeature, in association with detecting one or more categories ofidentity-correlated objects for reversible obscuring, a direct provisionof encryption parameters to the privacy protected image output logic410.

FIG. 5 shows a logic chart for a flow 500 of operations in a process inone or more methods for privacy-aware image capture in accordance withthe present disclosure. Description of certain examples of the flow 500operations, and aspects thereof, include references to FIG. 4 . Thereferences are not to be understood as limiting practices in accordancewith the flow 500 to FIG. 4 .

Example operations in the flow 500 can start upon receipt 502 of adigital image, e.g., an image IMG from the FIG. 4 imaging device 402.From receipt 502, the flow 500 can proceed to detection 504, whereapplied operations can detect whether the digital image includes one ormore categories of identity-correlated objects. The detection 504operations can be performed, for example, by the above-described system400 identity-correlated region detection logic 406. If the detection 504result is “No,” the flow 500 can return to 502. If the detection 504result is a positive result, e.g., “Yes,” the flow 500 can proceed toobscuring 506. Proceeding from detection 504 to obscuring 506 caninclude carrying information about the detected instances ofidentity-correlated object categories, such as category, and thelocation, and geometry of the detected instance. Examples are labeled inFIG. 5 , adjacent the flow arrow that extends from detection 504 toobscuring 506. Determination of such information is also described abovein reference to the identity-correlated region detection logic 406.

Obscuring 506 can include generating identity-correlated object categoryobscuring data, for example, based at least in part on categoryinformation from the detection 504. The flow 500 can then proceed fromthe obscuring 506 to privacy-protected image outputting 508.Privacy-protective image outputting 508 operations can includemodifying, e.g., obscuring the detected instances of identity-correlatedobject categories, in the original captured image, based at least inpart on detected and data generated in the obscuring 506.

One example obscuring 506 will be described assuming theidentity-correlated region detection logic 406 is configured to detectfaces as a first category object and license plate numbers as a secondcategory object. The example obscuring 506 will also assume aconfiguration in which faces are encrypted by an access key reversibleencryption, such as a public key encryption, and license plates areirreversibly obscured, e.g., Gaussian blurring, masked. In the example,operations in the obscuring 506 can include generating encryptionparameters and generating Gaussian blurring parameters, e.g., for use bythe obscuration logic 408. As one alternative, the identity-correlatedregion detection logic 406 can be configured to provide object categoryto the obscuration logic 408, and the obscuration logic 408 can beconfigured to provide only the obscuration type to the privacy protectedimage output logic 410.

FIG. 6 shows a functional block schematic of system 600 forprivacy-preserving image capture in accordance with the presentdisclosure. The system 600 includes an integrated imagecapture/obscuration device 602, that can include a housing that enclosesa device interior 602A. Supported within the device interior 602A can bea system for privacy-aware image capture in accordance with one or moredisclosed embodiments, such as the system 400. The system 600 can alsoinclude motor-driven mount 604 and can include an electronic imagetransmitter 606, configured to communicate the privacy protected imageto a receiver device that can be external to the housing.

FIG. 7 shows a simplified functional block diagram of a system 700 forprivacy-aware image capture in accordance with the present disclosure.Features of the system 700 can include key-access reversible obscuringof one or more categories of identity-correlated objects. To focusdescription on example key-access obscuring features, the FIG. 7implementation of system 700 is shown incorporating the system 400imaging device 402 (or receipt of images from an imaging device such as402), image buffer 404, and identity-correlated region detection logic406. The system 700 can include an encryption-based reversible obscuringlogic 702 (labeled on FIG. 7 as “EBR Reversible OBS Logic” 702) and caninclude an encryption key management logic 704 (labeled on FIG. 7 as“ENC-Key MNGT Logic” 704), and an encryption-based privacy image outputlogic 706 (labeled on FIG. 7 as “ENC PVI Generation Logic” 706).

The encryption key management logic 704 can be implemented, for example,as a distributed resource. As one non-limiting example, implementationof the encryption key management logic 704 can include key generationresources, or portions thereof, being local to, e.g., incorporatedwithin resources implementing the encryption-based reversible obscuringlogic 702. Such implementation can also include decryption key custodialor escrow resources such as, for example, can be maintained by thirdparties (e.g., an independent organization).

In an aspect, the encryption key management logic 704 can be configuredto generate a different encryption key each for new image IMG. Theencryption-based privacy image output logic 706 can be correspondinglyconfigured to apply that specific key to encrypt, i.e., effectuatereversible obscuring, of each identity correlation region. For example,assuming faces are the identity-correlated objects, a first key may beused for a first face while respective different keys used through “N”keys for respective “N” faces). The foregoing can be applied to othertypes of privacy related data. In other embodiments, different keys canbe used for different types of privacy related data. For example, afirst key can be used to encrypt facial data while another key can beused to encrypt license plate data. In this way, selective access torelevant privacy related data can be parceled out without giving accessto specified data.

FIGS. 8A, 8B, and 8C graphically show illustrative stages of receivingan image of a random subject person, applying various detection andobscuration data generating processes in accordance with thisdisclosure, and outputting a replacement, privacy preserving image ofthe subject person. FIG. 8A shows a graphic representation of a capturedimage of the random subject person, the image including a facepositioned 802 above an abstracted torso 804. Description will referenceFIG. 4 , assuming the system 400 is configured such that subjects' facesare identity-correlated objects and will reference FIG. 5 .

Referring to FIGS. 4, 5, and 8A, an image data corresponding to the FIG.8A can be stored in the image buffer 404 and, referenced to the flow500, operations can be at 506. Therefore, it will be assumed thatoperation by the identity-correlated region detection logic 406 detectedthe random subject person's face, and correspondingly provided detectioninformation to the obscuration logic 408. It will also be assumed thatthe obscuration logic 408, in response, generated an obscuration data,which is graphically represented as obscuration region or obscurationdata 806 in FIG. 8B (hereinafter “obscuration data”). The obscurationdata 806 can be, for example, an irreversible blurring or mask.Alternatively, the obscuration data 806 can be an access-key basedreversible obscuration. From generating the FIG. 8B obscuration data806, the system 400 can proceed to form and output a replacement image,such as the example in FIG. 8C, based at least in part on the FIG. 8Bobscuration data 806 and the original image in FIG. 8A. Referring toFIGS. 4 and 5 , such operations may be performed at 508, for example bythe privacy protected image output logic 410, with information from theobscuration logic 408.

As can be seen from FIG. 8B the obscuration data 806 corresponds to alocation, shape and dimension that approximately corresponds to thesubject's face. It will be understood that “approximately” meansaccurate to a degree such that recognition of the face has an acceptablelevel of difficulty.

FIGS. 9A, 9B, and 9C show illustrative stages of receiving an image of arandom subject person, and generating a replacement, privacy preservingimage of the subject person, applying example processes in accordancewith this disclosure. FIG. 9A shows the subject's face 902 positionedabove an abstracted torso 904, and name tag 906 that is on or affixed tothe subject's apparel. Referring to FIGS. 3, 4, and 9A, an image datacorresponding to the FIG. 9A can be stored in the image buffer 404 and,referenced to the flow 300, operations can be at 306. It will be assumedthat operations, e.g., by the identity-correlated region detection logic406, detected the face 902 and the name tag 906, categorized the faceas, for example, category 1 and the name tag as category 2. It will beassumed that the identity-correlated region detection logic 406 provideddetection information to the obscuration logic 408. It will also beassumed that the obscuration logic 408 in response generated, inaccordance with, e.g., flow 300 first type obscuring 308-1, a firstobscuration data 908 and generated, in accordance with, e.g., flow 300second type obscuring 308-2, a second obscuration data 910. The firstobscuration data 908 can be, for example, a first access-key basedreversible obscuration and the second obscuration data 910 can be, forexample, a second access-key based reversible obscuration. The system400 can then proceed to output, based at least in part on the FIG. 9Bfirst obscuration data 908, second obscuration data 910, and theoriginal image in FIG. 9A, a privacy-protecting replacement image, suchas the example shown in FIG. 9C. Referring to FIG. 4 , operations ofoutputting the replacement image may be performed by the privacyprotected image output logic 410 with information from the obscurationlogic 408.

FIG. 10A shows a functional block schematic of portions of anothersystem for privacy-aware image capture in accordance with the presentdisclosure. The FIG. 10A system is shown featuring a roadway observationimage capture device, implemented for purposes of example by the FIG. 6integrated image capture/obscuration device 602 described above. Withina field of view FOV of the integrated image capture/obscuration device602 is shown an automobile 1002, bearing a nameplate having a formlabeled 1004A and bearing a license plate number with a visible form1006A. For purposes of example, the automobile 1002 is also shownbearing a parking pass sticker having a visible form 1008A

The integrated image capture/obscuration device 602 in FIG. 10A can beconfigured to perform a flow such as shown in FIG. 3 . Referring to FIG.3 , categorizing 306, automobile nameplates will be assumed as a firstidentity-correlated object category, automobile license plate numbers asa second identity-correlated object category, and parking stickers as athird identity-correlated object category. Referring to FIG. 3 , logicflow blocks 308, the first type obscuring 308-1 will be assumed, forexample, a first type reversible obscuring, such as a public-keyencryption first type obscuring. The second type obscuring 308-2 will beassumed, for example, an irreversible obscuring, such as a Gaussianblurring, making, or replacement by a synthetic object of the detectedcategory. The third type obscuring, represented for example, by block308-N, will be assumed as another public key encryption.

FIG. 10B illustrates a privacy preservation replacement of the FIG. 10Aimage, applying the above-described first, second, and third obscuringtypes to the license plate, nameplate, and parking permit respectively.The privacy preservation replacement shows the visible first obscuredform of the license plate as a first cross-hatching 1004B, the visiblesecond obscured form of the nameplate as a second cross-hatching 1006B,and the visible third obscured form of the parking permit as a thirdcross-hatching 1008B. An entity that possesses the decryption key forone of the first type obscuring and second type obscuring will be ableto view a corresponding one of, but not both of, the original licenseplate form labeled 1004A and parking permit visible form 1008A. In thisexample, no entity will be able to view the original nameplate visibleform 1006A, as the second type obscuring was irreversible.

FIG. 11 shows a simplified functional block schematic of a computersystem 1100 on which aspects of systems and method in accordance withthe present disclosure can be practiced. An implementation of thecomputer system 1100 can include a processor 1102 and an instructionmemory 1104 that can be coupled to one another through a bus 1106.Implementations of the processor 1102 can include, but are not limitedto, ASIC (application-specific integrated circuit), FPGA (fieldprogrammable gate array), a generic-array of logic (GAL), and theirequivalents. The computer system 1100 can include a data memory 1108 anda large capacity storage 1110, each of which can be coupled, forexample, via the bus 1106 to the processor 1102. It will be understoodthat the instruction memory 1104 and the data memory 1108 are logicfunctions and can be implemented, for example, as respective resourcesof a shared memory resource.

The instruction memory 1104 and data memory 1108 can be implemented ascomputer readable, non-transitory storage media, (e.g., ROM (read-onlymemory), EPROM (electrically programmable read-only memory), EEPROM(electrically erasable programmable read-only memory), flash memory,static memory, DRAM (dynamic random-access memory), SRAM (staticrandom-access memory).

The computer system 1100 may be coupled, for example, via a networkcontroller 1112, to a network resource such as the WAN (wide areanetwork) 1114, such as the Internet or a local intranet. The computersystem 1100 can include an input/output interface 1116, which canprovide an input port for receiving image data from the integrated imagecapture/obscuration device 602. The input/output interface 1116 can alsoprovide interface, for example, to positioning actuators for theobscuration device 602. The computer system can also include a display1118.

The foregoing discussion discloses and describes merely exemplaryembodiments of an object of the present disclosure. As will beunderstood by those skilled in the art, an object of the presentdisclosure may be embodied in other specific forms without departingfrom the spirit or essential characteristics thereof. Accordingly, thepresent disclosure is intended to be illustrative, but not limiting ofthe scope of an object of the present disclosure as well as the claims.

Numerous modifications and variations on the present disclosure arepossible in light of the above teachings. It is therefore to beunderstood that within the scope of the appended claims, the disclosuremay be practiced otherwise than as specifically described herein.

CONCLUSION

Although the subject matter has been described in language specific toexample structural features and/or methodological steps, it is to beunderstood that the subject matter defined in the appended claims is notnecessarily limited to the specific features or steps described. Rather,the specific features and steps are disclosed as example forms ofimplementing the claimed subject matter.

What is claimed is:
 1. A system comprising: an image capture device, anidentity-correlated region detection logic, an obscuring logic, and aprivacy protected image output logic; the image capture deviceconfigured to receive an image; wherein the image is a still image, asequence of images, or a moving picture image; the identity-correlatedregion detection logic configured to: detect one or moreidentity-correlated object categories; determine category instanceinformation about said one or more identity-correlated objectcategories; the obscuring logic configured to generate obscuring databased at least in part on the above-described category instanceinformation from the identity-correlated region detection logic; theidentity-correlated region detection logic and obscuring logicconfigured to include an automatic category-specific selecting betweenirreversible obscuring and reversible obscuring; and privacy protectedimage output logic configured to form and output a replacement imagebased on the obscuration data and the image.
 2. The system of claim 1,wherein said object categories are selected from the list consistingessentially of: human faces, nametags, tattoos, and automobile licenseplate numbers.
 3. The system of claim 1, wherein said category instanceinformation is selected from the list consisting essentially of:location, shape, geometry, and dimension.
 4. The system of claim 1,wherein the detection logic is configured to detect a combination ofregions constituting personal identification information.
 5. The systemof claim 1, wherein the detection logic is configured to detectparticular positionings and patterns of constituent regions; and saidobscuring logic is configured to generate obscuring data comprising oneor more of the constituent regions that contain at least a partialobscuring of the particular positionings.
 6. The system of claim 1,wherein the obscuring logic is configured to receive the categoryinstance information from the identity-correlated region detectionlogic.
 7. The system of claim 1, wherein said obscuring logic isconfigured to generate obscuring data to function as selectors orinstructions for use by the privacy protected image output logic; theobscuring logic configured to include encryption-based reversibleobscuring data of one or more categories of identity-correlated regions.8. The system of claim 1, wherein the image is a frame; and saidobscuring logic is configured to obscure a corresponding region in anumber of frames succeeding and a number of frames preceding the frame.9. The system of claim 1, wherein the system is configured to output aprivacy protected image based on first obscuration data, secondobscuration data, and the image.
 10. The system of claim 9, comprising ahousing; wherein the obscuring logic and the image capture device arecontained within the housing; said housing configured to resist directaccess to the image capture device.
 11. The system of claim 10,comprising an electronic image transmitter configured to communicate theprivacy protected image to a receiver device external to the housing.12. A method for generating a privacy protected image comprising thesteps of: providing an image capture device, an identity-correlatedregion detection logic, an obscuring logic, and a privacy protectedimage output logic; receiving an image with the image capture device;wherein the image is a still image, a sequence of images, or a movingpicture image; detecting one or more identity-correlated objectcategories with the identity correlated region detection logic;determining category instance information about said one or moreidentity-correlated object categories with the identity correlatedregion detection logic; the obscuring logic generating obscuring databased at least in part on the above-described category instanceinformation from the identity-correlated region detection logic; thesteps of detecting and obscuring including automatic category-specificselecting between irreversible obscuring and reversible obscuring; andthe privacy protected image output logic generating an object category,generating an obscuration type; and generating the privacy protectedimage using the object category, obscuration type, and the image;wherein said privacy protected image output logic is configured to formand output a replacement image based on the obscuration data and theimage.
 13. The method of claim 12, wherein said object categories areselected from the list consisting essentially of: human faces, nametags,tattoos, and automobile license plate numbers.
 14. The method of claim12, wherein said category instance information is selected from the listconsisting essentially of: location, shape, geometry, and dimension. 15.The method of claim 14 comprising the step of a privacy protected imageoutput logic outputting a replacement image based on informationreceived from an obscuration logic.
 16. The method of claim 15,comprising a step of the obscuring logic including encryption-basedreversible obscuring into one or more categories of identity-correlatedregions.
 17. The method of claim 16, wherein the obscuring logicgenerates category-appropriate encryption parameters and to provide theparameters to the privacy protected image output logic.
 18. The methodof claim 12 comprising the step of the detection logic detecting acombination of regions constituting personal identification information.19. The method of claim 12 comprising the step of the detection logicdetecting particular positionings and patterns of constituent regions;and said obscuring logic is configured to generate obscuring datacomprising one or more of the constituent regions that contain at leasta partial obscuring of the particular positionings.
 20. The method ofclaim 12 comprising the step of the obscuring logic receiving thecategory instance information from the identity-correlated regiondetection logic.
 21. The method of claim 12 comprising the step of theobscuring logic obscuring data to function as selectors or instructionsfor use by the privacy protected image output logic; the obscuring logicconfigured to include encryption-based reversible obscuring data of oneor more categories of identity-correlated regions.
 22. The system ofclaim 12, wherein the obscuring logic is configured to includeencryption-based reversible obscuring into one or more categories ofidentity-correlated regions.
 23. The system of claim 22, wherein theobscuring logic is configured to generate category-appropriateencryption parameters and to provide the parameters to the privacyprotected image output logic.
 24. A system comprising: an image capturedevice, an identity-correlated region detection logic, an obscuringlogic, and a privacy protected image output logic; the image capturedevice configured to receive an image; wherein the image is a stillimage, a sequence of images, or a moving picture image; theidentity-correlated region detection logic configured to: detect one ormore identity-correlated object categories; determine category instanceinformation about said one or more identity-correlated objectcategories; the obscuring logic configured to generate obscuring databased at least in part on the above-described category instanceinformation from the identity-correlated region detection logic; privacyprotected image output logic configured to form and output a replacementimage based on the obscuration data and the image; wherein the system isconfigured to reversibly obscure at least one of the identity-correlatedobject categories and irreversibly obscure at least one of theidentity-correlated object categories.
 25. A method for generating aprivacy protected image comprising the steps of: providing an imagecapture device, an identity-correlated region detection logic, anobscuring logic, and a privacy protected image output logic; receivingan image with the image capture device; wherein the image is a stillimage, a sequence of images, or a moving picture image; detecting one ormore identity-correlated object categories with the identity correlatedregion detection logic; determining category instance information aboutsaid one or more identity-correlated object categories with the identitycorrelated region detection logic; the obscuring logic generatingobscuring data based at least in part on the above-described categoryinstance information from the identity-correlated region detectionlogic; the privacy protected image output logic generating an objectcategory, generating an obscuration type; and generating the privacyprotected image using the object category, obscuration type, and theimage; wherein said privacy protected image output logic is configuredto form and output a replacement image based on the obscuration data andthe image; and reversibly obscuring at least one of theidentity-correlated object categories and irreversibly obscuring atleast one of the identity-correlated object categories.